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Accountabilities for D & V Chatbot Operational Pilot 


e DAILY -All chats and feedback are reviewed daily by the dialogue team 
= Chatbot Specialist does daily audit of chats - updates flows and intents, 


Objective: p updates Visio - full day 
Increasing self-service = Chatbot and Content roles meet for 45 -60 min to review the chat 
opportunities while reducing the conversations and feedback: ee 

b f calls that t t * create notes in Kanban for opportunity, review and assignment 
num A OT Calis a 80 to m agen e Update flows and intents 
for Driver and Vehicle questions + WEEKLY 


=" SWAT has 2 @ 30 min meetings ensure alignment of information as well as 
eee sharing common barriers and potential fixes to connect with the responsible 
Limited Scope: 5 webpages areas. 


è i i it li e AS NEEDED 
Register Vehicle Permit Licence = Outreach to SWAT partners through Kanban or dedicated meeting 


Plate and Sticker dependant on item and urgency (60 min monthly) 
) = Issues creating new web content 
3 Buy or Sell Used Vehicle Ontario e New content errors - Connect with MGCS web 


` i lke |) F e MGCS web - form that requires full solution and content writing 

Renew Driver's Licence e ServiceOntario online tools - sharing with DAB for improvement 
e Driver's Licence e MTO - work with SME to determine page owner, policy, web, Drive Test or 

: : work through MGCS - some more successful 
e Payment Help: Driver & vehicle = New Content from Web 
i F e translated and approved - easily updated 

online services e COVID and resumption of business - changes take time 

e New product - if in scope create new flows taking time 
= Records and Information Freedom of Information 

e Working with MGCS RIM & FOI about record series and the new data 

directive, currently ongoing work 2 
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Chatbot Conversation — Two-year stickers? 


Chat hears it first — followed quickly by ServiceOntario Service Centres — changes made next day. 


Conversation in Chatbot Gutceme 

After the new launch of Renew Driver’s Licence and Stickers pages on July 30th, , We nina th with ou SNA 
our chats indicated confusion on the two year pricing which stated $140 and communication partners who worked 
should be $240 and was noted a SWAT. The ADMO of Customer Care was hearing With Cabinet. 


e This was corrected this next day by 


t h e Sa m e t h i n g a n d re a cl Your renewal fee may vary based on your situation, For example, the expiry date for your sticker has a 
changed. Cabinet 
Passenger vehicles 
Image of page: Region 1 year 2 year 
Southern Ontario $120 $140 
Northern Ontario $60 $120 
Other types of vehicle 
Type of vehicle Southern Northern 
Ontario Ontario 
Motorcycle $42 $21 
Commercial vehicle (gross weight 3,000 kg and less, $120 $120 
commercial use) 
Commercial vehicle (gross weight 3,000 kg and less, $120 $60 
personal use) 
Mopeds $12 $12 


Motorized snow vehicle $15 $0 


Mntarizari mahila hama £120 san 
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Chatbot Conversation — English and French mix 


Conversation in Chatbot 

Since the updates to the pages - A chat directed that French version of Outcome 

Renew your driver’s licence Renouveler un permis de conduire | 2 AIRES IE MSC cor a a 
. . 7 e This was corrected within the hour 

Ontario.ca contained some English. 

SWAT team communication partners responded quickly and it was 


updated in the hour. Renew your driver's licence 


Pour renouveler votre permis de conduire en ligne, il vous faut : 
| mage of page e le numéro de votre permis de conduire 


e le numéro a7 chiffres qui se trouve près du trille, au dos de votre permis de conduire 


e votre code postal 


Renouveler votre permis de conduire 


Renouvelez votre permis de conduire et carte Santé 


Vous pourriez étre en mesure de renouveler votre permis de conduire et votre carte Santé avec 
photo de l'Ontario si ces deux documents expirent en même temps. 


Renouvelez votre permis de conduire et carte Santé 


Si vous n'êtes pas admissible au renouvellement en ligne, présentez-vous à un centre ServiceOntario. 
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Chatbot Conversation — Lack of Clarity 


Here is a chat that helps us be better communicators and collaborators. 


Conversation in Chatbot - This is from the public in one 
paragraph — numbered by the client in the chat 


“I’m confused about whether my license is eligible to be renewed online. 


1. 


You notified me pre-pandemic that my photo needed to be updated 
when my license was due for renewal. 


During the pandemic, your website advises that clients would be notified 
when to renew and that extended expiry dates are in place for products 
expired on or after March 1, 2020 (my case). 


Your website cites that a full license is eligible for online renewal unless 
cancelled or suspended, on the first landing page; yet, when I click 
“Renew driver’s license” the second page indicates also that if the license 
is expired more than 12 months, then it’s not eligible. Please be 
consistent in your direction to clients. 


Your website further directs clients to avoid visiting a ServiceOntario 
centre unless absolutely necessary. 


There is a lack of clarity, mis-direction, and confusing messaging. 


Do | need to renew my license or do | trust there is an extended expiry 
date (unspecified) and I will be notified?” 


What did the SWAT team do? 


° After the new launch of Driver’s Licence Renewal 
webpage July 30, 2021 - the chats indicated confusion 
with information as early as August 5t" and share with 
MGCS Comms. 


° The Chatbot team were completing the comms change for 
when this feedback became a call to action — Submitted 
change and did a walkthrough August 10th 

° It highlighted many of the issues about the alignment 
between channels and Ministries 

e The page links to other webpages and the online 
application — there were 4 pages and 1 application to 
align 

e MGCS, MTO and ServiceOntario are working now to align 
all information 


° Since then there has been a resumption of business and 
teams have been stretched. Alignment is ongoing. 
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Since resumption 


MPBSD 23-071 Record 07 SOMTO-DVChatbot-Content Alignment Dec 


Page 7 of 14 


Chatbot Conversation — Clarity 1 - Service Finder 


Conversation in Chatbot 
After the new launch of Renew Driver’s Licence page, our chats indicated 
confusion about - Service Finder - Renew a driver's licence (gov.on.ca) 


Clients are confused between the messaging on the Renew Drivers Licence webpage 
and Service Finder Webpage — alignment is required about expiries over a year. 
Service Finder information is policy correct but not COVID correct — any links are 
updated, but not new content - information spans more than D&V 

Renew Drivers webpage offer the extended expiry and has removed the expired over a 
year information — that sends people incorrectly to Drive Test 

Public needs to know what do — they have been questioning this for a while now 


Summary and Images: 


Summary of details to align 
e a full licence (G, M class) that isn’t expired for oiio 
more than a year, suspended or cancelled 

e a driver's licence that does not require a test 
(vision, written or road) 

e you don't need new photo (check your renewal 
notice) 

e if your licence has been expired for over 1 


What you need to know before you start 


ou may aiso be able to renew your photo health card ai the same 
si Renew a driver's licence and photo health card online for more 


year you must visit a DriveTest centre Visit Driver's licence renewal & for more information 


Outcome 

e ODS is working with CMI in ServiceOntario to 
make changes — expected Mid-September 

e Service Finder (SF) holds the appointment 
booking for College Park and will be maintained 
until that changes 

e ODS is capturing the correct details so Service 
Finder redirects, our team review to ensure we’ve 
looked at all angles. 

e This ensures all information on the SF is 
directed to the correct information and will 
provide clarity to the public — ODS action 
outstanding. 


© Renew a driver's licence online + 


E Renew a driver's licence in person - 


$90.00 
Visa. MasterCard. D 


yobs, Cash 


What you need to know before you start 
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Chatbot Conversation — Clarity 2 - Applications 


APPLICATION - currently being investigated 


Twitter brings us back to 
conversations in Chatbot 
This tweet came with 
resumption and linked to the 
alignment in the works 
Image of page: 


viruw topics 


LordCrankypants @Crankyp... - 16h 
Replying to @ONtransport 

Any way you can waive the "must have 
expired within the last 90 days to 
renew online" caveat? Doesn't really do 
much to encourage your public health 
measures by creating a scenario where 
a ton of people are going to have to 
flock to @ServiceOntario locations to 
renew. 


AM a an M = 2 


You can renew your driver’s licence online if: 


eyou have a full licence (G, M class) that isn’t cancelled, suspended, or expired 


for more than 12 months 


Note: if your licence expired on or after March 1, 2020 (during the 


COVID extension), you 


eyou have a combination class of G1M, G2M, GM1, GM2 (only the full portion 


will be renewed online) 


may still be eligible to renew online 


eyou haven’t updated your address within the last 90 days 
eyou don’t need tests (vision, written, or road) 
eyou’re not removing a vision condition 


eyou have no new medical conditions that may prevent you from driving 


eyou don’t have outstanding fines 


WEBPAGE — updated information 


What you need to 
know before you 
renew 


Online renewal eligibility 
You can renew online if you: 
© have a full licence (G, M class) that isn't cancelled or suspended 


+ have a combination class of G1M, G2M, GM1, GM2, M1 or M2 (only the full portion will be 
renewed online) 


e haven't updated your address within the last 90 days 

o don’t need tests (vision, written, or road) 

e are not removing a vision condition 

© have no new medical conditions that may prevent you from driving 


e don't have outstanding fines or penalties 


Outcome 

e Digital Adoption Branch is 
reviewing the alignment from 
webpages to the application 
for clarity 

e MGCS Comms is offering 
support and following up 
No update on changes yet 
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Chatbot Conversation — Clarity 3 - MTO Resumption 


WEBPAGE -— Driving and roads | Ontario.ca 


Web Opportunities and Issues 
e No link to MTO renewal requirements which is a very current 
topic 
e Links to CAB Reopening Ontario | Ontario.ca which has no link to 
renewals requirements 
Link Wording MTO 
e Renew a licence that will expire soon or a licence that expired less 
than 1 year ago 
* More than one year changes to before March 1, 2020 
e Renew a licence that expired more than 1 year ago 
e More than one year changes to before March 1, 2020 
* Content 
e sends people to DriveTest 
e Expired over a year to March 2020 and update the tests 
+ Updated last August 5th 


à 


Ontario @ 


RSS 


Driving and roads 


Information about road conditions, driver's licences, vehicle registration, emissions testing, 
and commercial vehicles. 


COVID-19 information for truck drivers 


Driver licensing 
Licensed drivers 
© Renewe licenc 


Penalized drivers 
© Learn 


Renewal requirements for driver, vehicle and carrier products and other documents extended due to COVID-19 | Ontario.ca 


Renew an expired driver’s licence 
Content sends people to DriveTest 
¢ If your driver’s licence is expired or 
cancelled for more than 1 year, 
you cannot renew online. Instead you 
must: 
visit a DriveTest centre or book an 
appointment at ServiceOntario 


Bay and College (Toronto 


e Tests you may have to take 
If you have a full driver’s licence (class G 
or GM) that has been expired or 


cancelled for more than 1 year, you must 
pass some tests before you can renew. If 
your licence has been expired or 
cancelled for: 

1-3yrs, 3-10 yrs and over 10yrs 
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Chatbot Conversation — Clarity 4- MTO Resumption 


WEBPAGE - https://www.ontario.ca/page/ministry-transportation 


Web Opportunities and Issues 

e No link to MTO renewal requirements which is a very current topic 

e Links to CAB Reopening Ontario | Ontario.ca which has no link to 
renewals requirements 

e Links to SO Register a vehicle (permit, licence plate and sticker 


Ontario.ca 
e Missing the link to the temporary permit which is a temporary = R : 
. 5 Ministry of Transportation 
licence plate sticker ig ee ean arty ere A EAO EESO E 


quality of life and support a globally competitive economy. 


Walki ling and driving Vehicles and traffic Transportation planning and 
y information infrastructure 


What we do 
e Bulld then 


anaso? 
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Chatbot Conversation — Clarity 5 - SO Resumption 


WEBPAGE -— ServiceOntario | Ontario.ca 

¢ Links to SO right up front focus- easy 

e New webpages do not focus on 
deadline dates and do not reference 
expiry “since March 1, 2021” 

e Renew adriver’s licence | Ontario.ca 

e Renew a licence plate sticker 
Ontario.ca 


Web Opportunities and Issues 

e 2clicks to link to MTO renewal 
requirements which is a very current 
topic — this is the only place to find 
these details 

e Links to CAB Reopening Ontario 
Ontario.ca which has no link to 
renewals requirements 


à 


Ontario @ 


ServiceOntario 


At ServiceOntario you can get your driver's licence, plate stickers, health cards, birth 


certificates, register a birth and more. 


Renew your products online 
Over 40 services are available online making it fast, easy, 
and convenient to renew now. 
Here are some of our top online services, renew: 
e 2 plate sticker 
© your driver's licence 
e an accessible parking permit 
e an Ontario Photo Card 
e your heaith card 


Learn more about our COVID-19 response 


Drivers Vehicles 


First-time driver: get a licence 


Buy/sell a used vehic! 


Register a vehicle (plate, permit sticker) 


ce plate sticker renewal | Online 
Renew a drivers licence: outside Ontario r r 

renewal: outside 
Pay defaulted driver fines |Online 


Change address: drivers licence Temporary licence plate stici 


ca Used vehicle information package (UVIP) 


Online 


Change information on a vehicie permit 


Locations, hours and contact 


Forms and publications 
Service Canada 


Employment Insurance 


Pa 


Insurance Number 


Renewal requirements for driver, vehicle and carrier products and other documents extended due to COVID-19 | Ontario.ca 


ServiceOntario: COVID-19 (Coronavirus) response 


Find out when you need to renew your products by and how product delivery is affected. 


Ifyou do visit a ServiceOntario centre. you must wear a face mask or 2 face covering inside at all times to help stop the spread of COVID-19 
(coronavirus). Learn m 


ut face coverings and face masks. 


On this page 


vel Coronavirus (COVID-19) 


nded expiry date 


2019 Novel Coronavirus (COVID-19) Related 


To help ensure your safety and the safety of others, please consider completing your transactions 
online (if available) and staying home if you are sick 


We appreciate your patience during this time as the situation continues to evolve. 


Extended expiry date 


If you did not renew your licence plate sticker(s) during the COVID-19 extension, you are required 
to pay for that skipped renewal fee and your next renewal fee. Find out more information about 
extensions, 


We are ending the expiry date extension period for product(s) that expired on or after March 1. 2020, 
Dates you need to renew by: 


February 28, 2022 


© licence plate sticker (renew now) 
© Note: heavy and commercial vehicles 


(cenew now) 
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Chatbot Conversation — Clarity 6 - MTO Resumption 


WEBPAGE - Fees mis-understood as and seen as unfair 


Passenger Vehicle, Light-Duty Truck and Motorcycle Owners 


Licence plate stickers for all other vehicles must be renewed by February 28, 2022. 


If you did not renew your licence plate sticker(s) during the Extended Validity period, you will be 
required to pay the renewal fee for the previous year(s) and the current year renewal fee. 


For example, if your licence plate sticker expired on April 15, 2020, and you decided to renew it on 
September 15, 2021, you will have to: 
e pay for your sticker to cover the time from April 16, 2020, to April 15, 2021 


e pay for the future one or two year period, which began on April 16, 2021 


Web Statement 

If you did not renew your licence plate sticker(s) during the Extended 
Validity period, you will be required to pay the renewal fee for the previous 
year(s) and the current year renewal fee. 


Chatbot Statement Fix 
You will be required to pay the renewal fee for the previous year(s) and 
the current year renewal fee. 


WEBPAGE - Novice is New Driver 


Novice drivers (Class G1, G2, M1, or M2) 


Novice drivers with licences that would have expired between March 1, 2020, and December 31, 2022, 
will have until December 31, 2022, to complete testing to maintain or upgrade their licence. 


Regular expiry dates for Class G1 and G2 novice driver's licences will resume January 1, 2023. 


If you enrolled in a Beginner Driver Education (BDE) course between March 1, 2019 and December 31, 


2021, will have until December 31, 2022, to complete the course. 


Web Statement 
Novice drivers (Class G1, G2, M1, or M2) 


Chatbot Statement Fix 
New (Novice) drivers (Class G1, G2, M1, or M2) 
¢ 3 languages used for G licensing 


e MTO was using Novice and New — Moved to NoviceNew 
matches ServiceOntario 
e Inclusive — New Driver for ESL 


MPBSD 23-071 Record 07 SOMTO-DVChatbot-Content Alignment Dec Page 13 of 14 


Feedback 
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Chatbot Conversation — Fastest and Easy 


Conversation in Chatbot 


“I'd like to send someone on my behalf to register my car get a sticker - | understand there's a form 
that must be filled out but can't find that form on this site. Can you help with that? Thank you. Wow 
you are fast!” 


e This person was provided with the “third party answer” we have created in the chatbot 
They did not have to navigate between pages — we can use this as an opportunity to look at this 


information on the webpage 
e They were able to self-serve and it was fast!! 


Chatbot Feedback 


“Fastest service | ever got - it was so easy. Thank you for the help.” (Rated a 5) 


